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ABSTRACT 

The workshops associated with the Evaluation for 
Individualized Instruction Project are planned to provide teachers an 
opportunity to improve their skills in construction and development 
of evaluation procedures. This is based on the premise that one 
successful initial approach to the improvement and individualization 
of instruction is through the improvement of teacher-made tests. 
Presented in this operational guide for teacher workshops are eleven 
Instructional Sessions, they are: One — Educational Objectives; 

Two — Defining Global Objectives; Three — Behavioral Objectives; 

Four — Hierarchy of Learning Behaviors; Five — Purposes of Achievement 
Testing; Six — Techniques of Measurement; seven — Multiple-Choice 
Items; Eight — Basic Statistical Measures; Nine — Test Analysis 
Interpretation of Computer Output; Ten — Validity and Reliability; and 
Eleven^— The Liaison Role of the Workshop Participant. A list of 
reference materials is included. (LS) 
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Introduction 



I. NEED FOR THE PROJECT 

The need for this project arises from a number of closely associated ideas* These ideas are all related to 
important instructional tasks undertaken consistently and frequently by classroom teachers. 

One such idea is that it is unfortunate that instruction is limited by tests that are narrow in scope and 
improperly conceived* A second idea is a conviction that teachers can build suitable tests that measure 
much more than a low-level recall of information. The third idea is that teachers can utilize test data in 
the process of individualizing instruction. The fourth idea is that students can profit from receiving feed* 
back about their learning. The fifth idea is that teachers, and indeed all others interested in the educa* 
tional process, need to know how the pupils are performing more often than once each year. 

The workshops associated with the Evaluation for Individualized Instruction Project are planned to 
provide teachers 3n opportunity to improve their skills in construction and development of evaluation 
procedures. This is based on the premise that one successful initial approach to the improvement and in- 
dividualization of instruction is through the improvement of teacher-made tests* 

These project workshops should produce ( f ) a group of teachers who are sophisticated in the con- 
struction of evaluative instruments and in the techniques for using test data to improve the individual* 
Ization of instruction; (2) an available pool of teacher-made test items that are appropriate for measuring 
both information and the broader aspects of education; (3) the recommendations to students of remedial 
procedures to be followed if tested skills and understandings have not been mastered* 

To attain these ends, immediate as well as long-range objectives for the workshop have been devel- 
oped* These are stated in terms of the terminal behavior of the participants* 
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GLOBAL WORKSHOP OBJECTIVES 



A. IMMEDIATE OBJECTIVES 

The immediate instructional objectives of the workshop are those which can be evaluated as each 
workshop progresses and immediately after termination of the workshop, 

During the course of the workshop and as a result of instruction received, the teacher can 

1* distinguish between properly and improperly stated instructional objectives. 

2. write instructional objectives stated in terms of student behavior and applied to a par* 
ticular area of content. 

3. criticize multiple-choice items in terms of the anatomy, grammar, and logic of the form 
as well as the plausibility, clearness and accuracy of the content. 

4. write well-formed multiple*choice test items which test behaviors critical to attain* 
pient of important instructional objectives. 

5* demonstrete understanding that the objectives of instruction should include attainment 
of cognitive behaviors above the level of rote recall by writing multiple-choice items that 
test, not only recall, but also higher levels of cognitive behavior such as "comprehension, 
understanding, application, analysis, synthesis/'^ 

6. participate in the compilation of a complete list of content specified for areas of ap* 
plication and employed by the member school districts. 

B. LONG-TERM OBJECTIVES 

The long*term instructional objectives of the workshop are those which depend on implementation 
by teachers in their districts over an extended length of time. 

h 

As a result of participation in the Evaluation Project work^bp teachers can 

1. recall thatlho three major purposes of the Evaluation Project are 

a. to provide teachers with improved skills in the construction, development, and use 
of classroom evaluation procedures. 

b. to demonstrate that such improvement in teacher skill will lead to the improvement 
of programs of individualized instruction to promote mastery of essential objectives. 

c. to provide a model for the use of continuous evaluation information in both improving 
instruction and in giving information to communities regarding the progress of 
students in their schools. 

2. demonstrate their understanding of the three major purposes of the Evaluation Project 
by 



a. answering questions about the project. 

b. demonstrating and teaching the skills of objective item writing. 
^Taxonomy of Educational Objectives, Benjamin S. Bloom, et at. 
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c. explaining uses of the item pool and of the information that can be gained by use of 
data from item pool tests. 

d. using the item pool themselves to construct tests in their own subject area. 

e. using the results of item pool tests to analyze their own Instructional programs and 
pupil needs. 

f. revising their own instruction in the light of information gained through statistical 
analysis of test results so that 

(1) classroom instruction is more effective. 

(2) individual pupil needs are better understood and are more adequately mot 

3. demonstrate their knowledge and understandingof their "liaison" role In the Evolu* 

* ation Project by 

a, reporting in writing to their school districts-summarizlng their workshop activities 
and detailing ways in which they believe skills they have gained in the workshop 
can be used to help improve evaluation procedures in their districts. 

b, assisting in changing evaluation procedures where requested by their district admin- 
istration. 

c, requesting assistance from the Evaluation Project Staff in planning and carrying out 
a program of information and education. 

d, using the techniques for assisting innovation learned in the Evaluation Project work- 
shop. 

4, demonstrate their appreciation of the values of the project by 

a. participating with positive enthusiasm in the workshop projects. 

b. sharing ideasand offering suggestions during conferences and in general sessions of 
the workshop. 

c. attending voluntarily, special meetings and offering their services outside their own 
districts to help other participants where such assistance is requested and the dis- 
tricts approve the cooperative effort 

d. providing a personal mode! of proper use of evaluation tools and techniques in their 
own subject areas by drawing upon the item pool for the construction of tests and 

by using the results to evaluate on-going instruction and to meet individual pupil need 
in the classroom. 
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PLAN OF OPERATION: EVALUATION PROJECT WORKSHOPS 



The plan of operation of the workshops intersperses instructional sessions, led by one or more of the 
project staff, with work sessions in which the participants write and edit instructional objectives and 
multiple-choice items. Simultaneously, the staff maintains contact with former workshop participants 
by supplying them with materials and observing evaluation techniques employed in their classes. 

A. INSTRUCTIONAL SESSIONS: Teachers receive instruction in (1) the basic concepts 
of evaluation and measurement, (2) the writing of instructional objectives, (3) the 
writing of well-formed multiple-choice test items, (4) the construction of balanced 
classroom tests, (5) the analysis of test results, and (6) the application of test 
analysis to classroom instruction. Especially stressed are the benefits to individual 
students resulting from the use of test data to plan remediation, enrichment and 
acceleration. 

The instructional content of the workshops is carefully planned but the approach is 
. informal. Teachers are encouraged to question the information given, to seek clarifi* 

* cation and to explore problems in relation to the concepts being taught. 

This portion of the workshop is evaluated by means of a multiple-choice post test, 
and by sampling from the objectives and items written by teachers throughout the 
workshop sessions. The objectives and items used to evaluate instruction and 
teacher progress are checked against a list of errors similar to that used by the 
teachers to edit their items. 

B. WORK SESSIONS: Teachers write behaviorally stated instructional objectives and 
test items. Staff associates work closely with participants, helping them to analyze 
and edit the form and content of the objectives and items they write. 

C. LIAISON: The role of the workshop teacher in assisting change in the school is ex- 
plored. Teachers learn that even after completing the workshop, they may expect the 

. continuing support of the project staff in (t) providing up-to-date information on 
the project, (2) helping them to fulfill the expectations of their school and district 
with regard to the objectives of the project 

Workshop teachers are requested to report in writing to their district, summarizing 
their work as a participant in the project. They are asked to describe the ways they 
think the skills they have acquired by participating in the Evaluation Project can be 
used to produce better learning in their schools. 

A meeting Is held near the end of each workshop. This meeting brings together pre- 
vious and current workshop participants. In this way, ail participants are kept up-to- 
date on the project and the meeting also provides a valuable opportunity for sharing 
ideas. 
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Instructional Session One: 



Educational Objectives 



\ 



A. OBJECTIVES 
The participant 

1. can demonstrate knowledge of educational objectives by identifying specific changes 
in their form over the past century. 

2* can comprehend tha relationship between objectives, evaluation and learning activities by 
giving specific examples of each and describing their interdependence. 

3. can state the purpose of objectives in evaluating the effectiveness of innovative programs, 

4. can demonstrate knowledge of the sources of educational objectives as specified by T yler by 
naming the particular source considered in an illustrative example. 

B. INSTRUCTIONAL PROCEDURES: 

t. Lecture on the history of educational objectives 

2. Discussion of the purpose of educational objectives 

3. Lecture on the relationships between objectives, learning activities, and evaluation 

C. PARTICIPANT'S PREPARATION: 

1. Part i of Tal mage's Instructional Objectives 

2. Tyler's Basic Principles of Curriculum and Instruction, pp. 3-28 

D. INSTRUCTIONAL PROCEDURE: Discussion of the sources of educational objectives 

E. MATERIALS: 

1. Instructional Cycle: chart on the relationships between objectives, learning activities, 
and evaluation. 

2. Part l of Talmage's Instructional Objectives 

3. Tyler's Basic P ri nciples of Cu rriculum and instruction, pp. 3-2B 

F. J EVALUATION: Proficiency Quiz No, 1 
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STUDY GUIDE - SOURCES OF CURRICULUM 



A. Miss Raven is beginning her career as a mathematics teacher in an economically disadvantaged area 
school. Approximately five percent of the school's graduates continue their education. She has an 
excellent series of units on calculus that she designed while student teaching in a wealthy suburban 
community. Since she had such a fine response to these materials while student teaching, she decides 
to use them in her present situation. The children do very poorly with them. Miss Raven is convinced 
that they could master the material if her students would "only study." Which of the following 
sources of curriculum was probably being ignored? 

1) subject matter specialists 

2) children's needs 

3) desires of the local community 

B. Give a reason for your choice. 
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EVOLUTION OF EDUCATIONAL OBJECTIVES IN THE U. S. SINCE 1900. 



1900 - W. W. I 



1918 - 1925 



1930's - Present 



The mind was thought to have certain faculties, such as memory and reason, 
which could be trained or disciplined by proper exercise. 



Objectives were stated in terms of^jie^ental faculty to be trained. The study of 
Latin was considered especialiy/as its mastery required memory, while its grammati- 
cal structure provided exercise in orderly reasoning. 

Learning was perceived as the acquisition of highly specific patterns of behavior. 
Each task was viewed as a stimulus to which the student was to learn the appropri- 
ate response. 

Examples: 

1. Able to add 2 + 3 

2. Able to add 2 + 4 

3. Abletoadd2 + 5 

4. Able to spell believe 

5. Able to spell receive 



Difficulties: Number of objectives for each curricular area was phenomenal. One 
course in arithmetic listed 3,000 objectives. 

This view of objectives declined after 1 926 for two reasons: 

1. Teacher could not keep in piind so many objectives. 

2. The approach denied the development of generalized behavior 

The Eight-Year Study sharply articulated the idea that learning can best be defined 
as a change in pupil behavior and therefore, objectives should be directed towards 
obtaining this change. 



Examples: 

1. The Development of Effective Ways of Thinking 

2. The Dr^elopment of Effective Work Habits and Skills 

3. Increased Sensitivity to Social Problems 

4. Greater Awareness of Esthetic Experiences 

5. An Appreciation of Man's Relation to his Physical Environment 



Difficulties: Objectives of this type are usually too genera! to provide direction 

to the classroom teacher and yet they are still found frequently in curriculum guides. 



Mid-Century Committee on Outcomes in Elementary Education 
"Behavior = Action by a Pupil" 



Examples: 



1. The pupil knows the customs of India. 

2. The pupil can use the library for obtaining further materials. 

3. The pupils enjoy participating in group activities. 

The. work of James Popham, Robert Magerand others 
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"Behavior - Obsen/able Action by a Pupil" 

Examples: 

1. The student will circle the names of the five most recent presidents 
of the United States when given a list of ten past presidents. 

2. The student will solve four of five multiplication problems with the 
product less than twenty- five. 

3. Given a set of arguments and conclusions, the student will correctly 
select 80% of those whose conclusions are warranted* 







Instructional Session Two: 



Defining Global Objectives 



A. OBJECTIVES 

The participant 

1. can describe the purpose of stating educational objectives in various forms, e.g., public 
relations, global, instructional. 

2. can, from a list of objectives, indicate which are globally stated and which are behavioral ly 
stated. 

3. can rewrite global objectives into a behavioraliy observable form. 

4. will listen opcnmindedly to the philosophy of stating objectives behavioraliy as evi- 
denced by remarks made in a discussion on the topic. 

5. will attempt to write behavioraliy stated objectives that help to measure the attainment 
of a global objective. 

6. will react in a rational, knowledgeable manner to objections to the philosophy of behav- 
ioral objectives as evidenced by rebuttals to stated arguments. 

7. will accept change in student behavior as a measure of attainment of objectives rather than 
subjective opinions, as evidenced by the participant creating problematic situations and 
making judgments about proficiency based on performance. 

B. INSTRUCTIONAL PROCEDURES; 

1. Participants will complete a List of Selected Global and Behavioral Objectives. 

2. Discussion of the necessity of defining objectives in terms of pupil behavior if they are to 
provide direction to the teacher in guiding learning activities. 

C. PARTICIPANT'S PREPARATION: Part II ofTalmage's Instructional Objectives 

0. INSTRUCTIONAL PROCEDURES; 

1. Provide practice in defining global objectives in terms of student behavior. 

2. Provide practice in rewriting global objectives in behavioral terms. 

3. Group critique of efforts to rewrite global objectives. 

E. MATERIALS: 

1. Part II of Talmage's Instructional Objectives 

2. List of Selected Global and Behavioral Objectives 

F. EVALUATION: Proficiency Quiz No. 2. 



A LIST OF SELECTED GLOBAL AND BEHAVIORAL OBJECTIVES 



Place an X before the objectives which you feel would provide valuable guidance to you in planning 
learning activities to realize them 

1. The student will be able to correctly thread a sewing machine. 

2. The student will develop a sense of the cultural unity of man. 

3. The student will list and describe the themes of Shelley's poams. 

4 The child will develop interest in music. 

5. . The child will demonstrate an interest in Mark Twain's works by reading and re- 

porting on novels or short stories not required in class. 

6. 1 The student will up '^rstand fundamental concepts of physics. 

7. The pupil will recognize the incorrect usage of capital letters when he employs 

rules given in class. 

8. The student will learn about collective-bargaining. 

9. The student will be able to write an essay in which he contrasts the arguments for 

having a democracy or a dictatorship. 

10. The student will be able to set up and conduct an experiment to determine the 

density of rocks. 

11. The pupil will demonstrate a real appreciation of the contributions made by the 

late Senator Everett Dirksen. 

12. The student will learn the parts of speech. 

13. — _ The pupil will be able to list the bones in the human foot. 

14. The child will analyze an argument by identifying assumptions, conclusions, and 

emotionally laden statements. 
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Instructional Session Three: 



Behavioral Objectives 



A. OBJECTIVES 
The participant 

1. can state at least throe reasons for restating global objectives into a behavioral form. 

2. can indicate from a list of verbs, which are directly observable and which must be observed 
indirectly. 

3. can indicate from a list of objectives, which are stated behavioral! / and which are not. 

4. can indicate, from a list of objectives, which are stated in terms of the student and which 
are stated in terms of the course or teacher. 

5. can indicate, from a list of objectives, those that are explicitly referent to a concept or con- 
tent and which are not. 

6. can rewrite imperfectly stated objectives into acceptable, behavioral form. 

7. can write behavioral objectives in his own grade and subject area, using the global objectives 
from curriculum materials as a guide. 

8. can state the three domains of student behavior (cognitive, affective and psychomotor) and 
give examples of each in behavioral form. 

9. can classify given behavioral objectives into cognitive, affective and psychomotor catego* 
ries. 

10. will place behavioral objectives in their proper perspective with respect to evaluation and 
learning activities as evidenced by remarks made in discussions. 

1 1. vyiil explain to others the philosophy and methodology of behavioral objectives upon com* 
pletion of the workshop and returning to his district. 

12. will encourage others to write behavioral objectives to evaluate pupil progress and inno- 
vative programs in his district as evidenced by the forming of program committees and re- 
quests for assistance from the project staff. 

B. INSTRUCTIONAL PROCEDUR ES 

1. View Popham's filmstrip. Instructional Objectives, 

2. Present Popham's paper. Instructional Objectives as a written record of the filmstrip 

presentation. ™~~~~ 

3. Discussion of the anatomy of a properly stated behavioral objective. 

4. Prepare and discuss lists of directly and indirectly observable verbs. 

5. Provide practice in rewriting behavioral objectives. 

6. Provide practice in writing behavioral objectives in the participant's own subject matter 
areas. 

7. Group critique of behavioral objectives written by participants. 
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t 

C. PARTICIPANT'S PREPARATION: Mager's Preparing Instructional Objectives. 

D. INSTRUCTIONAL PROCEDURES: Discussion of Mager's approach to writing behavioral 

objectives 

E. MATERIALS: 

1. Filmstrip, Instructional Objectives and answer sheets 

2. List of directly and indirectly observable verbs to be prepared by participants 

3. Anatomy of a Behavioral Objective 

4. Mager's Preparing Instructional Objectives 

F. EVALUATION: Proficiency Quiz No. 3 
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INSTRUCTIONAL OBJECTIVES* W. Jr.mesPopham 



This program deals with the manner in which those involved in teaching or the development of in- 
structional materials should specify their goals. The topic is one of paramount importance for the pro- 
spective product developer. 

As you go tnrough the program you will be asked to inspect certain frames, and then to respond on the 
answer .sheet which has been provided. Use a 5 x 7 card as a mask so that you do not inadvertently see the 
right answer which is presented below the three asterisks. Sometimes you will sec three asterisks in the 
middle of the page as this: 



* * *- 

When you do, use your card to cover the area below the asterisks until you have responded on the answer 
sheet, then check for the correct answer below the asterisks and proceed with the program. Now begin the 
formal part of the program. 

Pupils go to school to get an education. And most of us would agree on the basic aims of the American 
schools. We want to produce good citizens, competent workers, and sound thinkers. However, most 
educators realize that such "basic” aim:, are usually stated in terms so broad as to defy precise interpre- 
tation. If teachers wish to really implement these objectives they must have a more definite idea of what , 
the objective means. 

Teachers have always worried about the importance of instructional objectives, yet the kinds of ob- 
jectives which teachers have endorsed, usually made little difference in the nature of the instructional pro- 
gram. The principal reason for thL is that these objectives were stated in terms too broad and too am- 
biguous to allow anyone to agree upon what the objective meant. 

In the next few frames, you will see some examples of actual objectives appearing in courses of study 
or lesson plans in public school use today, 

"At the end of the course the student will have developed qualities of rational thought and good 
citizenship." 

For this first objec live try to decide what the objective means and what kind of an educational program 
would be built in order to achieve the objective. Do you think others viewing this objective would reach 
a similar conclusion regarding its meaning and an appropriate educational program? Answer yes or no by 
circling your answer on the answer sheet alongside Number 1. 

The answer is No, for the objective is broad and permits a variety of interpretations. How about this 
objective? 

'The student will gain an appreciation of the importance of Western European literature." 

What kind of educational program would be planned to achieve it and, more importantly, how could one 
tell whether or not the objective had been achieved? Do you think most people would agree on the mean- 
ing of this objective? Circle your answer alongside Number 2. 

* * * 



•With permission, the material in this program is reproduced with minor modifications from the script of the copy- 
righted filmstrip-audiotape program Educational Objectives, by W. James Popham, Vimcet Associates, Los Angeles, 
California, 1966. 
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The answer should be No, for although this objective limits the content to Western European literature, 
it is still extremely vague. Objectives such as these are not much help in planning instruction. Suppose 
a school staff wishes to determine systematically whether it has been successful in achieving this kind 
of objective. What types of cxar.vnations or what sort o; observations of the students might help in 
judging whether the objective has been achieved? The same question can be asked of the following ob- 
j actives- how can one tell whether th iy have been successfully achieved? 

The student will grasp t ha signif icance of the environment in which he lives. 

Students will become familiar with the basic concepts of biology. 

Do you think most educators would agree on the kinds of tests which would reflect satisfactory 
achievement of these goals? Answer on your answer sheet by Number 3. 

# # * 



Again the answer is No. Thei 3 would be great disagreement over the way to measure attainment of these 
objectives, 

Unless objectives unambiguously communicate what the educator attends to accomplish, they are of 
Julie instructional value. In fact, one might think of a continuum in which educational objectives become 
more useful as they become less ambiguous. 

But why are ambiguous objectives of little value to the teacher? Since educators have been writing 
vague objectives for so many years, maybe they are worthwhile. Undoubtedly, they have some public re- 
lations value in that the schools can communicate general aims to the public with such broad statements. 
Sut in the classroom, broad, vague objectives are next to worthless because a teacher really can't make 
am/ instructional plans from them. Objectives should allow the teacher to know where he is going, that is; 
.hey should permit the teacher to formulate instructional plans for himself and his students so that the ob- 
jective toil! be achieved. But if the objective is so vaguely stated that a number of interpretations is pos- 
sible, is the objective useful? Yet, for no good reason, vague objectives are popular with too many 
classroom teachers, 

For instance, many teachers and curriculum workers describe objectives in terms of concepts topics, or 
generalizations to be treated in the course. 

The course will cover: 

1. The Industrial Revolution. 

2. The Civil War. 

3. The Reconstruction Period. * 

These objectives are examples of such "content coverage" goals. But further consideration will reveal 
that the teacher does not merely wish to cover these topics-rather, the teacher expects that coverage to 
help the pupils become educated. Statements such as these objectives, then, really miss the point of 
education. 

Other teachers and curriculum workers may state objectives in terms of what the teacher is to do 
during the course, 

During this unit the teachr will: 

1. Analyze the causes of World War II. 

2. Discuss the background of the U.N. 

3. Point out the current perils to peace. 
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Yet, o*'*ce mare we runtime thi.t ihe teachir should really be interested in how his activities influence 
his pupils. Cf the only rsnl objective in a history class was for the teacher to analyze the causes of World 
War II, no pupils need be present. The teacher cculd "analyze" in an empty room. Such statements, 
thcT-fore, fail to provide the teacher with any explicit guidance about ways the instruction should affect 
the students. 

Some teachers rso state objectives in extremely bro.cd terms, with no specific reference. 

Tr.o students vvUh 

1. Develop appreciations, 

2. Increase his intercut. 

3. Develop conceptual thinking. 

These objectives, because cf their nonspecific nature give little help to the systematic planning of in- 
struction. Almost frothing eculd be done. 

What then is*a good instructional objective? How should instructional goals be stated so that they are 
of value to the teacher in acting learning activities and in subsequently evaluating to see whether the ob- 
jectives h.v;c been accomplished? Good instructional objectives must he stated in terms of student behav- 
icr. Yes, meaningful ond valuable instructional objective must be described by stating how the student , ( , 
behaves, or will be able to behave, after instruction. The more specifically the pupil behavior can be de- 
fined the better. For v.*h*n the teacher's goal to change the student'? observable behavior, a way of 
*u dr’.i.g whether the objective h?.a been achieved is p;ovided~n8mely, to observe whether the behavior 
change has occurred, but v.h?t kind of "behavior"? How should we describe the way o pupil will behave 
aucr instruction? la this a satisfactory description? 

The pupil will undsrsteruS. 

Wkyi a teacher says that a student will "understand," precisely what does he mean? For instance, 
think about the kind cf evidence you could use to demonstrate that this next objective had been achieved. 

Tl;i c-wrJjnt will understand the meaning of tho Monroe Doctrine;. 

Or this one? 

The 2 V*c!r.nt will :^r.!!y ur.dorsteod the meaning of the Monroe Doctrine. 

i t is true that most people have n somewhat similar conception of what the word "understand" means, 
but if you ask them to be more specific, for example, to describe the kind of student behavior, even test 
behavior, that reflects "understanding," *, ou will find tremendous differences in their interpretation of 
this term. 

For instance, seme would think the student understood the meaning of the Monroe Doctrine when he 
could write out a description of it from memory; or maybe when he could answer a series of true-false 
questions about its background; cr perhaps only when he could properly identify instances where the 
Doctrine was violated. 

It is clear that terms such as "understanding," and "knowledge," and "insight," allow considerable 
latitude with respect to their interpretation. Even though terms such as understanding refer to the stu- 
dent, it is next to impossible to tell what they mean unless one further specifies what type of student 
behavior signifies understanding. 

In the next frame, docs phrase A or phrase B allow for fewer interpretations? 
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A. To saiect answers correctly. 

B. To realize fully. 

Circle the correct answer on your sheet by Number 4. 



Phrase A is clearer because it describes an observable form of behavior. 

Phrase B describes a rather vague, internal type of response which, because it is unobservable, might be 
interpreted in many ways. 

In the next frame, pick the two phrases which identify a form of observable student behavior. 

At the end of the unit the student will be able; 

A. To differentiate. 

B. To enjoy. 

C. To comprehend. 

D. To construct 

Circle the appropriate two letters on your answer sheet by Number 5. 



A and D are the correct answers, for ''differentiating" and "constructing" identify behavior which is 
observable, while "enjoying" and "comprehending" refer to internal, unobservable responses of the 
individual. 

In this next frame select the two behavioral phrases and circle your answer sheet by Number 6 
accordingly. 

A. To think 

8. To repair 

C. To answer 

D. To appreciate 



B and C are the correct answers, for "repairing" and "answering" are behavioral, whereas "thinking" and 
"appreciation" are not at all specific. 

In the next frame, is objective A or B less ambiguous? 

A. The pupil will learn his multiplication tables. 

B. The pupil will solve 4 or 5 multiplication problems. 

Circle the correct answer by Number 7. 

* * * 



Objective B is clearer because solving is an identifiable form of pupil behavior. The pupil's responses on 
a test would serve as a behavioral index of his solutions, whereas learning is one of those non-behavioral 
words which is open to many different interpretations. 

Which of these objectives is the more specific? 
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V 



A. The student will know the novels of Faulkner. 

B. The student will describe three literary movements of the Romantic Period* 

Circle the best objective by Number 8 cn your answer sheet* 

* . * * 



Objective B is the more specific since it identifies the student behavior which is sought-namely, 
written description. Objective A hinges on the interpretation of "know" and this could be something as 
simple as merely listing the names of Faulkner's novels or as difficult as writing critical analysis. 

In the next three frames select the behaviorally stated objectives by circling the appropriate letter on 
your answer sheet. 

A. The student will ba able to match the names cf world leaders with their country. 

B. The concept of world unity will be treated. 

For this frame answer by Number 9. 

* * ¥■ 



In the preceding frame, the behaviorally stated objective is A. Choose the behaviorally stated objective 
for this next frame and answer by Number 10. f 

A. The student will have familiarity with the major stylistic schools in English verse. 

B. The student will bo able to write a 6 line poem which incorporates iambic pentameter. 

# # * 

Objective B is the correct answer for the frame. Now choose the best answer for this frame and 
answer by Number 1 1 on your answer sheet. 

A. The student will really want to become a good teacher. 

B. When given two objectives, the student will bo able to correctly identify the one which is properly 

stated. 

* # r. 



In this frame, objective B is correct, r ;r while it is important to want to become a good teacher, it is 
difficult to determine whether or not a student really wants to. On the other hand, identifying properly 
stated objectives, as in choice B, is observable behavior-and incidently, will contribute much toward one's 
becoming a good teacher. 

! Now that you have had considerable practice in identifying behaviorally stated objectives, hern is a 
more difficult task-tho writing of such objectives. 

Is the objective in this frame stated behaviorally? 

The pupil will know the names of the five most recent presidents of the United States. 

Circle the correct answer on your answer sheet by Number 12. 



Of course not! For while this is a rather limited objective, the verb "know" is ambiguous because there 
are a number of observable student behaviors which might be used to reflect such knowledge. Now, on 
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your answer sheet, change this objective so that it is behavioral. That is, write out an improved ob- 
jective which there is no ambiguity with respect to the behavior which reflects knowledge of the names 
of the five most recent presidents of the United States. 



* * # 

Look at your objective and decide whether there is an observable student behavior described. If the be- 
havior of the student can he observed while he demonstrates this knowledge or if the product of his know- 
ledge is observable, then the objective is acceptable, For example, 

The students will correctly name aloud the five most recent presidents of the United States. 

This objective describes a student behavior that can be observed as it takes place, and thus it would 
qualify as a behavioral objective. 

This next objective describes a product, the list of circled names, which could be observed. 

The student will circle tho names of the five most recent U. S. presidents from a list of all of our presidents. 

This too is a behavioral objective. Most objectives involving student test performance are of this type. 

Other acceptable modifications of the objective might be the following: 

The student will list in writing the names of the five most recent presidents of the United States. 

When presented with S multiple choice questions, each presenting one of the five most recent presidents 
and three distractors, the student will choose the correct answers. 

If your modified objective was similar to these, you have prepared an acceptable objective. Inciden- 
tally, if your behavioral objective requires student performance on a test, it is necessary to carefully de- 
scribe the type of test that will be used. If possible, examples of the test items should be given, for it may 
be tempting to avoid the difficult task of explicitly describing student behavior by hiding behind such 
vague objectives involving test performance. 

The next objective is definitely non-behavioral. Change it on your answer sheet so that it is behavioral. 

The student will gain a working knowledge of the RCA tape recorder. 

* * * 



To judge whether your modified objective is acceptable, first see if it describes an observable student 
act or an observable product of a student act. If so, it is a behavioral objective. For example, the following 
few altered objectives would be acceptable. 

The student, when presented with the RCA tape recorder and a blank reel of tape, can record his own 
voice and play it back. 

The student will list in writing the seven most important parts of the RCA taps recorder, and describe 
the use of each. 

The student will point out six control switches on the RCA tape recorder and orally describe the way 
each is used. 

If your objective is simi'ar to those, it can be considered a behavioral objective. 

Here is a broader non-behavioral objective. 

* j 

The pupil will become familiar with the background of World War II. 
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Change it o; • you* answer Jiiesc to tlsrt it is hshs-'ic >T.i. 

<- it * 

In the first place, you must ne v rralb.a how truri 1 ' choices r.r* uvcilablJ in Lohavicrizing such on 
objective. The number o« bd; jicrai changes which might be mode ere dmosi unlimited, For instance, 
a teacher could have tl.a studen*: r*o rrytw..,.j from merely ihting a fa-.v causes of World War 1 1 to more 
complex betVivicr such as writing a c'txnmr r*tcd research paper. To toil v+ioihcr your medified objective 
is prooerly stated, you will have to judyj whether the objectives rireribe:; an observable student behavior 
or on observable product of ctudan: behavior. If it does, your objective is acceptable. 

You should now beab’u to diseri miuat;: betvvt* t bch«.v. lorally and ron bchuvionlly stated objectives* 
You should also bo sb'e to ehc;.£v c non behavioral object!'. :* to a behavioral objective. Why should these 
skills be of any benefit to you? In ctbsr words, what ire the perticulnr advantages of behavioral objec- 
tive’ 7 

For 01 iO thin^, an instructor or product developer specifies his objectives in torms of student behavior* 
This enables him to sslet ‘ appropriate evaluation procedures, for there is lets ambiguity with respect to 
the meaning of the objective. 

In the same w. n, behavioral objectives make it far easier for the teacher or programmer to select suit- 
able learning activities for the das: since hi ;;n*rvs precisely what kind of student behavior he is attempt- 
ing to produce. 

Another eJvsntrge of objective is that, since they are stated so specifically, the instructor 

himself con decide just he / adequate h:^ in:tfuct:o.ia( objectives ar3. He can also call on a colleague to 
improve th\- duality of his object 2 * go. When objectives are stated so broadly, it may be impossible to im- 
prove them. Specificity enables one to evaluate whether he is pursuing the rig.it aims. 

Kero is another significant ahvantogj of behavioral objectives. They can be given to mature students 
in advance of tho instruction so that the student can focus his energies on relevant tasks. He can then 
avoid spending his time either mastering peripheral material or in trying to out-guess the instructor. It 
may taka several .anter.ces to state an objective clearly, but once this is done, both instructor and student 
can be guided properly. 

Finally, behavioral objectives make it possible for the teacher, and others, to evaluate instruction on 
the basic of whether the students accomplish the ini nded objectives, Wo longer need an instructor be 
evaluated on whether hr, ins a "plowing personality" or a "wholesome philosophy of life". By employing 
behavioral objectives, teachers can chart their instruction a 1 goals ai.d then go about accomplishing them. 
By gathering behavioral evidence, the teacher can tell whether ho should retain or modify his instructional 
approach, If, for example, students fail to behave the v.ay he wis'vs them to after instruction, he can 
change tho learning activities. If students do acquire the behavioral changes he wishes, then perhaps he 
should expand or intensify the nature cf his objectives. 

Certainly there am other factors involved in good instruction, but precise behavioral objectives can 
markedly help most instructors to empirically augment the quality of their teaching. The key to im- 
proved instruction is indcod Behavioral Objectives. 



*3 



21 



ANATOMY OF A BEHAVIORAL OBJECTIVE 



The most useful form for stating objectives is to express them in terms of the behavior (both cognitive 
and observable) to be developed in the student and the content area (subject matter or concept) to which 
this behavior is to apply* 

AUDIENCE 

The student can 

BEHAVIOR 

Cognitive (analyze, recall, apply, etc*) 

Observable (select, draw, match, write, etc*) 

CONTENT 

Subject Matter (Macbeth, lenses, NRA, equations, etc.) 

Concepts (definitions, principles, theorems, etc.) 

e*g. The student will analyze an argument by commenting on the validity of the supporting 
statements with respect to the conclusion. 



O 
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Instructional Session Four: 



Hierarchy of Learning Behaviors 



B. 

c. 

D. 

e. 



f. 



OBJECTIVES 
The participant 

1. can state the six levels of the Taxonomy of Educational Objectives a nd the major sub- 
categories of each* 

2* can order given behaviors in increasing mental complexity. * 

3. can identify the taxonomic level of given behavioral objectives* 

4* can write behavioral objectives requiring cognitive skills at all levels of the Taxonomy. 

PARTICIPANT'S PREPARATION; Talrnage's Instructional Objectives, Part HI 

INSTRUCTIONAL PROCEDURES: Discuss the ranking of selected verbs in order of their in- 

creasing mental complexity-use practice sheet on Classifying Cognitive Behaviors. 

PARTICIPANT'S PREPARATION: Bloom's Taxonomy of Educational Objectives, (Chapter I) 

— — • — — I 

INSTRUCTIONAL PROCEDURES: 

1* Discussion of Chapter One of Bloom's Taxonomy. 

2. Discussion of the six levels of the Taxonomy and examples of objectives of each. 

3* Provide opportunities for participants to write objectives from their subject matter area at 
the different levels of the Taxonomy* 

4. Group critique of behavioral objectives produced by procedure No. 3. 

MATERIALS: 

T Practice Sheet in Classifying Behaviors 
2* Bloom's Taxonomy of Educational Objectives 
3* Talrnage's Instructional Objectiv es, Part III 
EVALUATION: Proficiency Quiz No. 4. 
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TAXONOMY OF EDUCATIONAL OBJECTIVES 1 

Cognitive Domdn 

1.00 KNOWLEDGE 

1.10 Knowledge of Specifics 

1 . 1 1 Knowledge of Terminology 

1.12 Knowledge of Specific Facts 

1^0 Knowledge of Ways and Means of Dealing with Specifics 

1.21 Knowledge of Conventions 

1 .22 Knowledge of T rends and Sequences 
1.23.Knowledge of Classifications and Categories 

1.24 Knowledge of Criteria 

1.25 Knowledge of Methodology 

1.30 Knowledge of the Universals and Abstractions in a Field 

1.31 Knowledge of Principles and Generalizations 

1 .32 Knowledge of Theories and Structures 

2.00 COMPREHENSION 

2.10 Translation 

2.20 I nterpretation 

2.30 Extrapolation 

3.00 APPLICATION 

4.00 ANALYSIS 

4.10 Analysis of Elements 

4.20 Analysis of Relationships 

4.30 Analysis of Organizational Principles 

5.00 SYNTHESIS 

5.10 Production of a Unique Communication 

5.20 Production of a Plan, or Proposed Set of Operations 

5.30 Derivation of a Set of Abstract Relations 

6.00 EVALUATION 

6.10 Judgments in Terms of Internal Evidence 

6.20 Judgments in Terms of External Criteria 

^ Bloom, Benjamins., et al, Taxonomy of Educational Objectives, David McKay Co., Inc., New York, 1968. 
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SELECTED OBJECTIVES IN THE SOCIAL STUDIES - PRACTICE SHEET 



Directions: Classify each of the following objectives according to its proper taxonomic level. 

1. Student can identify terms related to finance. 

2. Studentcan use his knowledge of Washington's political philosophy as expressed in his 

foreign policy speeches to analyze a series of statements and select those most representa- 
tive of Washington's viewpoint. 

3. Student will demonstrate his knowledge of Washington's Farewell Address by answering 

questions implied but not directly stated. 

4. Student will demonstrate his knowledge of the term, "protective tariff," by applying it to 

current day situations. 

5. Student can examine a table presenting data on the Industrial Revolution and translate 

this information into verbal statements. 

6. Student can distinguish between "expansionist" and "anti-expansionist" statements con- 
tained in current periodicals. 

7. Student can distinguish between moral and practical arguments put forth by the South to 

defend the Institution of Slavery. 

8. Student will be able to analyze the various theories of the causes of American Revolution 

as propounded by various historians and evaluate their validity. 

9. Student will determine the implications of the attack on Fort Sumter by comparing it with 

a given hypothetical situation. 

10. Assuming that President Andrew Johnson had been removed from office, the student can 

predict the possible consequences of such an action to the federal government. 
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PRACTICE WORKSHEET 



Listed below are pairs of cognitive behaviors. Select the behavior that involves skills of greater mental 
complexity. 



1 . 


a. 


memorize 


7. 


a. interpret 




b. 


analyze 




b. synthesize 


2. 


a. 


identify 


8. 


a. classify 




b. 


conclude 




b. validate 


i 

3. 


a. 


relate 


9. 


a design 




b. 


assess 




b. organize 


4. 


a. 


infer 


10 . 


a. estimate 




b. 


recognize 




b. deduce 


5. 


a. 


conclude 


11. 


a. evaluate 




b. 


rephrase * 




b. differentiate 


6. 


a. 


recall 


12 . 


a select 




b. 


distinguish 




b. contrast 
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Instructional Session Five 



Purposes of Achievement Testing 



A. OBJECTIVES 



The participant 

1. can state the relationship between objectives, learning activities and evaluation. 

2. can state five purposes of achievement testing. 

3. can discuss three benefits of pre and post testing. 

4. can differentiate, in a given situation, whether the testing done is measurement or evalua' 
tion. 

5. can discuss how proper evaluation proceduresare crucial to individualized instruction 
programs. 

6. will indicate, in discussion, that assignment of grades is a minor purpose of achievement 
testing. 

7* will encourage other teachers in his district to utilize test results for purposes other than 
assigning a grade. 

B. INSTRUCTIONAL SESSION: Review chart presented in Instructional Session One. 

C. PARTICIPANT'S PREPARATION: Purposes of Achievement Testing. 

D. INSTRUCTIONAL SESSION: Discuss the purposes of achievement testing. 

E» MATERIALS: Purposes of Achievement Testing. 

F. EVALUATION: Proficiency Quiz No. 5. 



PURPOSES OF ACHIEVEMENT TESTING; 



1. Pre-testing helps you, the teacher, know where and how to begin. 

2. Pre-testing frequently identifies those students in need of remedial help. 

3. Pre-testing provides a basis for flexible grouping in particular skill areas. 

4. Pre-testing coupled with post-testing tells you, the teacher, how much your students have improved. 

5. Test information provides a basis for student self-assessment and motivation, 

6. Test scores provide a basis for normative comparison with other students and with community, state, 
or even national norms. 

7. Test information provides one means for communicating intelligently with parents about a child's pro- 
gress in terms of the child's capacity. 

8. Test data furnish a basis for detecting, and hence for attempting to remedy, certain weaknesses in the 
curriculum. 

9. To determine the level of student achievement in the subject matter of the course at some point in 
time, in order to assign a grade. 

10. To provide a continuous assessment of student learning throughout a definite time period, (such as one 
school year), in order to pinpoint the time when "changes" in learning start to take effect and when 
they peak. Also continuous assessment over a period of several school years may compare learning 
curves. 



30 



31 



Instructional Session Six 



Techniques of Measurement 



A. OBJECTIVES 
The participant 

1. can list three different techniques of evaluating pupil progress. 

2. can state advantages and limitations of the different techniques of evaluating pupil progress. 

3. can indicate from a list of scoring procedures which are objective and which are subjective* 

4. can name an evaluatory procedure appropriate for measuring a given type of behavior 
sought. 

5. can demonstrate knowledge that ail testing situations provide samples of student behavior 
rather than being completely comprehensive by selecting test items to measure skills within 
a given unit. 

6. can construct and use a table of specifications in achieving balance in a test. 

7. can analyze a standardized test by specifying the type of student behavior to be elicited by 
each item. 

8. can, given a portion of a standardized test, construct a table of specifications for it. 

9. will write test items or problematic situations to utilize the different methods of measure- 
ment to obtain valid samples of student behavior. 

B. PARTICIPANT'S PREPARATION; Measuring the Attainment of Educational Objectives 

a INSTRUCTIONAL SESSION: 

1.. Discuss objective and subjective scoring. 

2. Discuss testing as providing a sample of a student's total behavior pattern. 

3* Discuss the advantages and limitations of essay and objective tests. 

4* Group critique of the appropriateness of test situations to measure certain types of be- 
havior using Selecting the Appropriate Measurement Technique work sheet as a guide. 

5. Group examination and discussionof the physics test and the accompanying Table of 
Specifications. 

D. PARTICIPANT’S PREPARATION: Examine Iowa Test of Basic Skills and T eacher's Manual. 

E* INSTRUCTIONAL SESSION: 

1. Delineation of the student behavior attempted t be elicited by each test Item, 

2. Provide practice in plotting student behaviors and items on Table of Specifications, 

3. Discuss the usefulness of the Table of Specifications for obtaining balance in a test. 

4. Discussion of the probable effectiveness of I.T.B.S. in eliciting these behaviors. 
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F. 



MATERIALS: 




1. Measuring the Attainment of Educational Objectives 

2. Iowa Test of Basic Skills and Teacher's Manual 

3. Selecting the Most Appropriate Measurement Technique-Work Sheet 

4. Table of Specifications 

5. Physics test and the chart plotting it on a Table of Specif ications 

6. Assembling theTest 

■ G* • EVALUATION: Proficiency Quiz No. 6. 
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MEASURING THE ATTAINMENT OF EDUCATIONAL OBJECTIVES 



L Written Tests 

A. ADVANTAGES AND LIMITATIONS 

1. Characteristics of Essay Tests 

a. The student produces and organizes his own answers. 

b. The answer is in the student's words. 

c. The test is limited to a few items. 

d. Answers can be judged as to the degree of correctness. 

2. Characteristics of Objective T ests 

.a. The task ishighfstructured. 

b. The student selects from given alternatives. 

c. There is a large sample of items. 

d. One uses a predetermined key. 

B. WRITING AND SCORING ESSAY ITEMS 

1. Writing Essay Items ( ’ ' 

a. Have in mind the mental process you want to be elicited by the student. 

b. Start with phrases like "compare,” "contrast,” "present arguments for and against," 

"give examples of." 

c. Define the task clearly. 

d. In achievement tests, do not use "in your opinion," etc, 

e. Beware of giving too many items in the time allowed. 

f. Do not mix essay and objective items when time is limited. 

g. Do not offer a choice of questions to be answered. 

2. Scoring Essay Items 

a. Decide in advance what is to be measured. 

b. If more, than one quality is being evaluated, give separate appraisals. 

c. Prepare a model answer with points to be covered and credit distribution. 

d. Read ail answers to one question before going on. 

e. Grade as anonymously as possible. 

f. More than one rater improves reliability. 

In subjective evaluation, the score assigned is determined by the personal opinion and judgment of the 
scorer, It is not determined by a prescribed scoring key or by the specifications in detail of the requirements 
for answers which willreceive various scores. Objective evaluation utilizes a simple predetermined list of 
correct answers so that opinion or judgment in the scoring procedure is eliminated. The scoring of multiple 
choice, true-falseand matching items is objective. Short answers or open-ended questions are partly ob- 
jective, partly subjective and essay items are usually totally subjective. 

C. WRITING AND SCORING OBJECTIVE ITEMS 
1. General 

a. Keep the reading difficulty low (unless you are measuring reading or verbal skills). 
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b. If opinion or authority is cited, indicate WHOSE opinion or WHAT authority. 

c. Be sure one item does not provide cues to another. 

d. Avoid interdependent items. 

e. Mak2 the correct answer vary randomly among the choices in a set of items. 

f. Avoid trick questions. 

g. Avoid ambiguity. 

h. Avoid trivia. 

2. True or False 



a. Be sure the item can be unequivocal and classified as true or false. 

b. Avoid specific determiners. 

c; Avoid indefinite terms of degree (most, some, few). 

d. Avoid negative statements. 

e. Keep the item to one idea. 

f. Avoid items where the answer depends on one insignificant word, phrase or letter. 

g. Avoid clueing by length of the item. 



3. Short Answer 

a. Avoid indefinite or open items. 

b. Omit only key words. 

c. Avoid a large number of blanks. 

d. Blanks are better placed at the end than the beginning. 

e. If a numerical answer is required, indicate the units in which it is to be expressed. 



4. Matching 

a. Items in a set should be homogeneous. 

b. The number of answers should be greater than the number of problems. 

c. The set should be relatively short. 

d. Answer choices should be in a logical order, if one exists. 

e. Directions should specify the basis for matching. 

f. Directions should indicate if an answer can be used more than once. 



5. Multiple Choice 



a. 

b. 

c. 

d. 

e. 

f. 

g. 



The lead or stem should clearly formulate the problem. 

Include as much as possible in the stem but avoid irrelevant material in the stem. 

Be sure there is only one correct or best choice. 

Items measuring application, etc. should be phrased in novel terms. 

Avoid grammatical inconsistencies between the stem and alternatives. 

Avoid "none of the above," "all of the above," "B & C," etc. unless they are absolutely 
necessary. 

If the negative is necessary in the stem, capitalize or underline the "not" or "except", 
etc. 



I! OralTests 

* .. ’i* ■ ’ ■ 

Because it must be done individually, oral testing is demanding in terms of teacher time and classroom 
management. Nevertheless, oral testing is necessary where learners cannot read or have handicaps that do 
not allow them to write or mark answers. 
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III. Performance T ests 



i 



This form of testing presents the student with a real situation and rates him on the procedure he uses 
to obtain a product or solve a problem, on the product or problem solution itself, or both. In some areas, 
such as physical education, vocal music and the dance, form is an aspect of performance important both 
to procedure and to product Teachers of language, debate and speech may test performance in many 
situations. The ultimate test of learning in English Composition, for example, is the ability to write a 
passable composition in English. Instruction in French, Latin and Spanish aim at the spontaneous gener- 
ation by students of acceptable sentences either orally or in writing. 



"v-A 






■> 




■V 
■ :v 



; S 



I 







■:{; 

a 

. V 
’ 1.1 

% 

;•! 




o 

me 




35 





3 

* 

J 







SELECTING THE MOST APPROPRIATE MEASUREMENT TECHNIQUE Exercise Sheet 



1. A chemistry teacher desirfes to measure his students' proficiency in the techniques of setting up a 
laboratory experiment Describe the technique you believe would provide him with the most valid 
appraisal of their ability to do this task: 



2. A social studies teacher is interested in measuring her pupils' progress in learning the skills connected 
with determining latitude and longitude. Describe the procedures that you believe wou Id provide the most 
adequate index of their growth: 



3. A language arts teacher is interested in measuring her pupils ability to compare and contrast the 
styles used by selected authors to achieve a certain reader response. What measurement techniques 
•* might be most appropriate to provide an index of the pupils' ability to do this task? 



i 
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4, A social studies teacher desires to measure his pupils* mastery of selected historical concepts. His 
groups* skills to engage in reading and writing is very limited. What measurement technique might 
give the most valid estimate of their mastery of the concepts? 



5, A science teacher wants to determine the level of his pupils* mastery of certain basic scientific data. 
Describe the techniques that he might use to obtain indications of their mastery of the basic data. 
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TABLE OF SPECIFICATIONS 



Instructional 
Objectives for: 
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BEHAVIOR VS. CONTENT 





Knowledge of 
facts 
and 

Terminology 


Understanding 
and Interpreting 
Graphical 
Material 


Applying 
Principles 
to explain 
Phenomena 


Recognizing 

Limitations 

of 

Principles 


Evaluating 
Statements by 
Scientific 
Method 


The Nature and 
Propagation of 
Light 

06) ‘ 


1, 2, 3 
5, 7, 13 
14, 15 




21, 23 


33 


43, 44 
45, 47 
50 


Reflection arid 
Refraction at a 
Plane Surface 
0) 


17 


41, 42 


20, 24 
30 


31. 35 


48 »J '• 


Reflection and 
Refraction at a 
Single Surface 
04) 


6, 8, 9 
10, 11, 12 
16 


39, 40 


22, 25 
28 


32 


49 


Lenses 

(ID 


4, 18 


36, 37 
38 


19, 26 
27, 29 
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(18) 



(71 



( 12 ) 



(5) 



( 8 ) 



PHYSICS TEST - OPTICS 



1. The velocity of light in mjles per second is about 

a. 3 x 10 6 . 

*b. 186,000. 

c. 93,000,000. 

d. 8,000. 

2. The astronomer who first discovered that light had a definite velocity was 

a. Galileo. 

b. Kepler. 

c. Newton. 

*d. Roemer. . 

e. Hale. 

J 

3. Light waves resemble sound waves in that both 

a. are longitudinal. 

b. are transverse. 

c: can travel through a vacuum. 

# d. are forms of energy. 

4. In a double concave lens, the image is 

a. erect, virtual and enlarged, 

b. inverted, real and diminished. 

c. erect, real and diminished. 

*d. erect, virtual and diminished. 

5. Scientists now regard the principle of rectilinear propagation of light as 

a. entirely disproved. 

b. established without reservation. 

*c. valid for many practical purposes. 

d. valid in so few cases as to render it useless. 

e. of very little importance. 

6. The focal length of a convex mirror 

a. is equal to R, the radius. 

*b. is always negative. 

c. is equal to 2R. 

d. is related to the aperture. 

e. is the point where all virtual images occur. 

7. The property that light as a particle and light as a wave have In common is 

*a. reflection. 

b. diffraction. 

c. interference. 

d. infraction. 
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In the next five questions, identify the parts of the figure labeled by choosing the corresponding letter 
beside the correct name. 



8. *b 

9. # c 

10. # d 

11. *e 

12. *a 




13. The ratio of the velocity of light in a vacuum to its velocity in any other medium is called the 

a. index of reflection. 

*b, index of refraction. 

c. index of diffraction. 

d. velocity ratio. 

e. ratio of Interference. 

14. The section marked B in the figure below is the 



a. corona 

b. shade 

# c. penumbra 

d. reflection 

e. umbra 




15. Which part of the eye corresponds to the adjustable diaphragm of a camera? 



a. pupil 

b. lens 

c. eyebrow 
# d. iris 

e. cornea 



16. The failure of parallel rays to converge at the focal point of a spherical m»rror is called 

a. astigmatism. 

b. declination. 

c. chromatic dispet 'on. 

# d. spherical aberration. 

17. The angle between a r.jy of light striking a surface and the normal to that surface at the point is the 

a. angle of coincidence. 

# b, angle of incidence. 

c. critical angle. 

d. angle of total reflection. 
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18. In the figure below, K is called the 



a. focal point. 

b. lens centroid. 

*c. optical center. 

d. optical focus. 

19. To determine quickly the 
upon a white screen. 

*a. far 

b. near 

c. large 

d. small 

e. light 

20. To hit a stick under water one should aim 

a. directly at the stick. 

b. in front of it. 

# c. below it. 

d. above it 

e. behind it. 

21. An eclipse of the moon occurs when 

*a. the earth is between the sun and the moon. 

b. thr moon is between the earth and the sun. 

c. the sun and moon are at right angles with respect to the earth. 

22. Given two examples of diffuse reflection, which is correct? 

*a. A 

b. B 

c. A and B 

d. neither A or B 

23. An airplane flying at a bright of 100 ft. casts a shadow on the ground v/hich is a 

shadow of the same plane flying at 10,000 feet. 

a. much smaller than 
b % much larger than 
*c. essentially the same size as 

24. The angle between the incident ray and a plane mirror is 25°. The angle between the incident ray and 
the reflected ray in this case is 

a. 65°. 

b. 50°. 

c. 115°. 

*d. 130°. 
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focal length of a convex lens, one can focus light from a very object 
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a. has f f in air. 

b. hasf - f in air. 

*c. has f< fin air. 

d. becomes a diverging fens. 

e. has no. longer any focusing power. 

27. A convex lens formed by placing two watch glasses (each with parallel sides) edge to edge, taped to- 
gether to make it airtight, if placed under water 

a. will still be converging lens. 

b. will not act like a lens at ali. | 

*c. will act lil'a a diverging lens. 

J. will let rays pass through it unaffected. 



28. A boy in front of a cylindrical mirror finds his image is 




a. Q- 

b. 6/2 
*c. 29- 

d. 40- 
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31. Since a periscope depends on the relative refraction indexes of glass and air, does a periscope work if 
totally submerged? i.c., can you still see underwater with it? 



a. No, because the water would bend the light r^ys out of range of the second prism. 

b. Yes, because refraction in a periscope is unaffected by the two media. 

c. No, because there is not enough light to cause sharp images. 

*d. Yes, because the only refracting is done within the periscope where air is still the medium. 
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33. A pinhole camera does not work when 

a. the object is too far away. 

# b. the hole is made wider. 

c. the hole is not centered in the camera. 

d. the screen is black. 

34. If two lenses have identical shapes and sizes, 

a. f is independent of n. 

bf r f 2 -‘ 

c. < n 2 implies <f 2 « 

# d. >n 2 irrlpliesf j <f^ 

35. Two boys standing on a beach with a straight shoreline, try to make the beam of a flashlight held by 
one reflect off the still water back to the other. 

a. It is possible to accomplish this because i = r. 

b. It is possible because light travels in straight lines. 

c. It is impossible because i £r. 

# d. It is impossible because the incident ray, the reflected ray and the normal must he in the same 
plane. 



Each question is answered by one of the five graphs below, For n - e, the horizontal axis = S Q 




An object is placed at the principle focus of a converging lens and moved slowly from the lens. 

36. Which graph could represent the location of the image as the object moves away? *d 

37. Which graph indicates the product of S Q and S- as the object moves away? *e 

38. Which graph could repre^nt the height of the image as the object moves away? *d 

* 39 The object is now placed at the principle focus of a parabolic mirror and moved away from the mirror; 
which graph could represent the ratio h Q / hj as the object moves away? # c 

40. Suppose an object moves along the principle axis toward the principle focus of a parabolic mirror at 
a speed of 1 m/sec. Which graph would represent the speed with which the image would move away 
from the principle focus an a function of time? "a 
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41. In the figure, angle A is c jSled 
a the reflected angle, 
b. the normal angle. 

*c, the critical angle, 
d. the acute angle. 



42. Which graph is correct for reflection? 




incident 

angle 




For Questions 43 - 50 express your evaluation of the statement by choosing the corresponding number 
of the correct stage. 

a. Belief Stage 

b. Authoritative Opinion Stage 
c Observation Stage 

d. Controlled Experiment Stage 

43. The moon reflects light from the sun. *c 

44. White light is a combination of the different colors of the sun's spectrum. *d 

45. Light travels from place to place instantaneously *a 

46. Convex lenses magnify small objects. # d 

47. The speed of light is the fastest thing obtainable in the universe. *b 

48. The angle of incidence equals the angle of reflection. *d 

49. A ray of light is bent more by glass than water. * d 

50. Light is a particle. 
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ASSEMBLING THE TEST 



LENGTH - 


The test should be long enough to sample significant behaviors called for by 
each objective, but short enough so that the student can complete it com- 
fortably within the time allotted. 


ITEM ORDER - 


One useful way to order test items is in accordance with the sequence of the 
content itself. Also, items nr.ay be arranged in the order of their perceived 
difficulty. 


KEY - 


Prepare the answer key before the test is duplicated so that the cutrect re- 
sponses can be randomly keyed. 


DIRECTIONS - 


Keep directions simple and explicit. 


LANGUAGE - . 


Language should bo direct and precise. The student should be able to answer 
on the basis of his information and understanding of the subject matter alone. 


APPEARANCE — 


Avoid overcrowded, or poorly organized format. Be sure the test is proofread 
for errors before the student gets it and that the errors are corrected. 


ITEM TYPE - 


The type of item to be used should be carefully selected so that it is the best 
possible way of measuring the degree of pupil attainment of each instructional 
objective. No more than two or three different kinds of items should be used 
in one test and they should be grouped together. 



Tbt above list by no means exhausts the important factors in putting together a good objective test. 

You will probably think of others. These suggestions were taken from Green, John A., T eacher-Made T ests . 
Harper and Rowe, New York, 1963 and Gorow, Frank F., Better Classroom T esting, Chandler Publishing, 
San Francisco, 1966. 
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Instructional Session Seven: 



Multiple - Choice Items 



A. OBJECTIVES 
The participant 

1. can list at least three advantages and two limitations of multiple-choice items. 

2. can, given a population of students, state the appropriate number of alternatives for items 
to be presented. 

3. can state three advantages of objective scoring as it relates to multiple-choice items. 

4. can state the definition cf terms used in discussing multiple-choice items, e.g., stem, foil, etc. 

5. can, given a multiple-choice item, identify the different parts. 

6. can, given the different forms of multiple-choice items, name each. 

7. can identify faults in given multiple-choice items. 

8. can rewrite correctly "wrongly stated" multiple-choice items. 

9. can write multiple-choice items that are free of item faults. 

10. can write multiple-choice items that measure cognitive processes more complex than simple 
recall of specif ic facts. 

11. can write multiple-choice items that require the student to perform a task critical to the 
performance of the objective it was written to test. 

12. will listen openmindedly to the advantages and methods of writing multiple-choice items 
as evidenced by participation in discussion. 

13. will provide evidence of believing that multiple-choice itemscan be used to test higher 
mental processes than recall of specific facts by actually attempting to write such items. 

14. will express to others the advantages and scope of objective testing, especially with multiple- 
choice items. 

15. will encourage others to use multiple-choice items in evaluating the degree of attainment of 
behavioral ly stated objectives. 

B. PARTICIPANT'S PREPARATION: Advantages and Limitations of Multiple-Choice Item.*. 

C. INSTRUCTIONAL CESSION: 

1. Discuss the Advantages and Limitations of Multiple-Choice Items. 

2. Distinguish clearly between product and process. 

D. PARTICIPANT'S PREPARATION: Anatomy of the Multiple-Choice Item. 

E. INSTRUCTIONAL SESSION: Review briefly the material presented in The Anatomy of the 

Multiple-Choice Item. 

F. PARTICIPANT'S PREPARATION: 



1. Typical Item Faults 

2* Examples of Item Faults — Practice Sheet 

6 INSTRUCTIONAL SESSION: 

1 . Rev iew participant's responses on the Example of I tern Fau Its practice sheet 

2* Group critique participants' efforts to correct item "faults" on Practice Sheet 

3* Provide practice in writing multiple-choice items for the behavioral objectives written by the 
participants in Instructional Session Three, B-6. 

4. Group critique items produced by participants through activity No. 3. 

H* MATERIALS: 

1. Advantages and Limitations of Multiple-Choice Items. 

2. Anatomy of the Multiple-Choice Item 

3. Typical Item Faults 

4* Examples of Item Faults — Practice Sheet 
I. EVALUATION: Proficiency Quiz No. 7. 
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ADVANTAGES AND LIMITATIONS OF MULTIPLE-CHOICE ITEMS 



Well constructed multiple-choice items have certain advantages for the student and the teacher as a 
means of measuring the attainment of instructional objectives. 

A. ADVANTAGES OF MULTIPLE— CHOICE ITEMS: 

L The student must demonstrate the specific ability the item calls for In order to select the 
correct response. He cannot avoid direct confrontation with the question* 

2. Multiple-choice items may be objectively scored since such factors as skill in verbal expres- 
sion, or speed and legibility of handwriting do not affect the teacher's assessment of the 
student's response. Only the response to tho question enters into the score. 

3. Since multiple-choice items are relatively brief, they allow a wide sampling of behaviors in 
• a short time. Writing (composing and recording) answers is generally conceded to be more 

time-consuming than reading and selecting an answer. 

4. Each multiple-choice item has one, stated, correct answer* Thus, scoring can be done by 
almost anyone, using a prepr/ed key. 

5. Scoring of multiple-choice items is fast and easy because the student's response is simple 
z, d unambiguous. Multiple-choice items may be machine scored, 

6* Multiple-choice items can be analyzed statistically with much greater ease than free re- 
sponse items. 

7. Many kinds and levels of learning behavior may be tested by multiple-choice items. 

8. The multiple-choice item generally provides greater test reliability per item than the true- 
false item. 

B. LIMITATIONS OF MULTIPLE-CHOICE ITEM: 

7. Handwriting skills, performances on musical instruments and similar skills and abilities 
should not be tested with multiple-choice kinds of tests. 

2. Although multiple-choice items can efficaciously measure the higher mental abilities, con- 
siderable caution must be exercised to insure that the cause of the pupil's lack of ability to 
engage in the higher mental processes as measured by the individual test item is not just the 
forgetting o' certain facts. This limitation can be overcome by two methods: 

a. Grouping items according to varying mental complexities on a specific topic so a more 
adequsre diagnosis can be made of the pupil's real problem. 

b. Including the necessary information in the test situation to insure that the pupil's lack 
of ability to engage in the higher mental processes is not strictly the lack of knowledge 
of selected facts. 



Ebel, Robert L., Measuring Educational Achievement. Englewood Cliffs, New Jor:,;y. Prentice-Hall, Ina. 1965 
Furst, Edward J., Constructing Evaluation Instruments, New York, David McKay Company. Inc., 1958, 
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THE ANATOMY OF THE MULTIPLE— CHOICE ITEM 



The multiple-choice item is an example of a supplied-response test. Each multiple-choice item has 
two (2) parts. These are: 

A. STEM OR LEAD 

1, Directions are a part of the stem or lead. Sometimes the lead is preceded by directions 
telling what to do in order to respond correctly. 

2. The stem or lead is of two main types. 

a. The direct (or bona fide) question 

Example - Why do living organisms need oxygen? 

b. The incomplete statement 

Example * During swallowing, the epiglottis covers the opening of the larynx so 
that 

Note: This type of lead is more difficult to write correctly but if it is well 

formed, it usually results in a more concise and pointed lead. 

B. ALTERNATIVES 

These alternatives are often called choices, options, or answers. 

1, One answer is clearly best and is the correct response. 

2. The “crippled'' or inferior alternatives are called distractors (or decoys or foils) and are 
incorrect answers. 

3, The alternatives may be expressed as words, clauses, sentences, numbers, pictures, graphs, 
diagrams and formulae; in short, anything that satisfies the logical requirements of the 
question posed is allowable. 

4. There are usually 4 or 5 alternatives. Items appropriate for the primary and intermediate 
grades may offer only 3 alternatives. 
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TYPICAL ITEM FAULTS - Revised Material 



1. Pupil behavior attempted to be elicited by the item is not congruent with behavior stated in the 
objective. 

For Example: Objective: The student can recognize sketches, drawings, or pictures of the past five 

presidents.of the U. S. 

Test Item: Which of the four men listed below K/aJnever President of the U. S.? 

a. George Washington 

b. Robert Kennedy 

c. Lyndon Johnson 

d. Harry S. Truman 

2. Item's stem does not present a single, definite, important question or problem, 

3. Item's alternatives do not include a single, correct, or best answer or completion, 

4. If the form of the stem is an incomplete statement, it should contain as much of the problem as 
possible. 

For example: Washington 

a. was a great general. 

b. was a plantation owner. 

c. was the first President of the U. S. 

Improved: George Washington is often referred to as the "Father of our Country" because he was 

a. a great general. 

b. a plantation owner 

c. the first President of the U. S. 

5. Item's alternatives are not stated as briefly as possible. 

6. Question or problem is hot stated in positive terms. (If it is necessary to use a negative, it should be 
underlined and/or capitalized, e.g. NOT, EXCEPT, UNlikely.) 

7. Alternatives are not grammatically consistent with the lead. 

8. Alternatives are not sufficiently similar in grammatical construction, degree of precision, or length. 

9. Alternatives are not Plausible, incorrect responses should tempt pupils who have incomplete or super* 
ficial knowledge of the material, 

10. Alternatives contain a "trick" word in an otherwise correct response just for the purpose of catching 
the reader. 
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EXAMPLES OF ITEM FAULTS - PRACTICE SHEET (Revised Materials) 



Each of the following items contain one or more of the faults listed on the sheet, Typical Item Faults. 
You are to do two things with each of the items: 

a. List the number(s) of the fault(sh 

b. Correct the item so that it is acceptable. 

t. Which of the following has helped most to increase the average length of human life? 

a. fast driving 

b. avoidance of over-eating 

c. wider use of vitamins 

d. wider use of inoculations 

2. At the Constitutional Convention, the "great 1 ' compromise 

a. gave small and large states equal representation in the Senate. 

b. made slave holding illegal. 

c. was opposed by Washington. 

d. gave western lands claimed by the states to the federal government. 

3. The part of speech that tells "how", "when", or "where" is an 

a. noun 

b. pronoun 

c. adverb 

d. conjunction 

e. verb 

4. The framers of the Constitution faced many problems. The delegates to the Constitutional Con- 
vention represented states with different interests and the delegates from the individual states 
wanted to see that their states' interests were protected. However, the delegates agreed that the 
Articles of Confederation needed to be changed in order to provide for 

a. a president whom everyone could respect. 

b. a stronger central government. 

c. a better understanding between states. 

d. a government for and by the people. 

5. Which one of the following leaders of the Revolutionary War did not want the Articles of Con- 
federation changed? 

a. Benjamin Franklin 

b. George Washington 

c. Alexander Hamilton 

d. Patrick Henry 
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6. The Chicago Symphony 

a. presents the largest number of concerts, 
b* has the largest number of players. 

c. has the largest number of male players, 
d* has the largest number of female players. 

7. Slavery was first started 

a* at Jamestown settlement. 

b. at Plymouth settlement* 

c. at the settlement of Rhode Island 
d* a decade before the Civil War 

8. Which event in the following list has been of the greatest importance in American history? 

a. Braddock's defeat 

b. Burr's conspiracy 

c* The Hayes-Tilden contest 

d. The Webster-Hayne debate 

9. The adoption of the Constitution was generally opposed by people who 

a. owed money 

b* thought that most people were unfit to govern themselves 

c. owned businesses 

d* thought the states should be destroyed 

tO* The student knows that ( 1 ) the Constitution provides for a census every ten years and (2) the reason 
that a provision was made for a regular national census. 

A census every ten years was provided for in the Constitution in Article I, Section 

a. 1 

b. 2 

c. 3 
d* 4 
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Instructional Session Eight: 



Basic Statistical Measures 



A. OBJECTIVES 

The participant 

1. can demonstrate knowledge of the definitions of certain statistical terms (mean, median, 
mode, skewness, kurtesis, normal distribution, standard deviation, and correlation) by 
matching correct pairs from lists of terms and definitions. 

' 4 * i 

2. can calculate for a given distribution of a small sample, the mean, mode, median, variance 
and standard deviation. 

3. can determine whether a given distribution is negatively or positively kurtotic. 

4. can determine whether a given distribution is negatively or positively skewed. 

5. can, by giving examples, demonstrate a knowledge of the implications of positive, negative, 
and zero correlation. 

6. can, by giving an illustrative example, show that correlation does not necessarily imply 
causality. 

B. PARTICIPANT'S PREPARATION: Basic Statistical Measures - Numerical Example (2 pages) 

C. INSTRUCTIONAL SESSION: 

1. Discuss definition of frequency distribution and histogram. 

2. Discuss problems in selecting intervals and the normal curve. 

3. Discuss definitions and formulae for mode, median, mean, variance and standard 
deviation. 

4. Calculate the actual values for the given data for each term in 3. 

D. PARTICIPANT'S PREPARATION: Basic Statistical Measures - Samples of Skewness and Kurtosis 

E. INSTRUCTIONAL SESSION 

1. Discuss positive and n^g-itive skewness (definition and example) 

2. Discuss positive and negative kurtosis (definition and example) 

3. Discuss meaning of significant skewness or kurtosis relative to normal curve 

F. PARTICIPANT'S PRErARATION: Standard Score Scales 

G. INSTRUCTIONAL SESSION 

1. Discuss role of standard scores, in terms of standard deviations and the advantages over raw 
score reporting. 

2. Discuss the relation of some of the familiar scales to standard scores. 

H. PARTICIPANT'S PREPARATION: 

1, Basic Statistical Measures • Examples of Correlation 
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2. Scores For a Fifth Grade Class (2 pages) 



I. INSTRUCTIONAL SESSION 

1. Discuss correlation as a measure of relation between 



and negative correlation. 



two variables; meaning of positive 



2. Discuss the notion that correlation does not ii 



not imply causality. 




>n, discuss the calculations that produce 
intuitive feelings about the variables. 



MATERIALS: 



1. Basic Statistical Measures • Numerical Examples 

2. Basic Statistical Measures • Samples of Skewness and Kurtosis 

3. Basic Statistical Measures - Examples of Correlation 

4. Standard Score Scales 

5. Scores for a Fifth Grade Class 

6. Glossary of Measurement Terms 
J. PROFICIENCY QUIZ NO. 8 
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FREQUENCY DISTRIBUTION 



The frequency distribution of a set of data (e.g, # test scores) can be represented in a variety of ways. 
Two are pictured in the NUMERICAL EXAMPLE, The first is merely to state the number of scores 
that occur in a set of intervals of scores where the intervals are determined by the number and range of 
scores. The second way to represent a frequency distribution is l / means of a histogram or bar graph. 
The vertical axis represents the number of scores, while the horizontal axis is the score itself or intervals 
of scores. 

THE NORMAL CURVE 

When the frequency distribution has a bell-like shape centered about the mean, it is said to be 
"normal". (It is unfortunate that the word "normal" carries connotations that are irrelevant to statistics.) 

Norma! distributions are very common. One has only to consider amounts like the number of leaves 
on oak trees versus the number of oak trees to conclude that many trees have nearly the same number of 
leaves as the average number. Few have f e h/ leaves or an enormously large number. I ntelligence 
is another example with great numbers of people near 100 and few people at the extreme ends of the 
scale. 

No classroom teacher should be satisfied with her accomplishments if her students* test scores are 
distributed normally. 1 1 is much more desirable to have a large proportion of students at the upper end 
of the scale which would result in a shape far from belMike. 

STANDARD DEVIATION 

Knowing only the mean or average of a set of scores does not. allow us to make any judgments about 
differences among the subjects tested; i,e., is the group fairly homogeneous or not? Som e measure of 
the spread of scores is necessary to answer such a question. One measure of spread is the standard 
deviation. 

A seemingly naive approach to measuring the spread of test scores would be to take the average dis- 
tance of each test score from the mean. If the scores differ greatly, this value would be large. If the 
scores were quite similar, the result would be small. 

For example, consider the following scores for nine students on a ten-item test. 

1,3, 4, 4, 6, 6, 6, 7, 8 

The average of these scores is 5. Now let us calculate the distance from each score to the mean. 

1 - 5 ® -4 

3- 5=-2 

4 — 5=1 —1 

4- 5 = -1 

6-5= 1 

6-5= 1 

6- 5= 1 

7- 5= 2 

8- 5= 3 

Now we shall take the sum of these and find the average “distance" from the mean. 

Sum = 0 0/9 = 0 
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